Filter files early in bytestream uploader to avoid redundant work and double handling #23252
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Re-implementation of #20575 and a test case for minimal mode.
In this PR;
--remote_build_event_upload=minimal
(default) are treated similar to those affected by the tagno-remote-cache
(they are omitted).The actual paths are still resolvable by the returned resolver (
bytestream://
URIs that do not reflect an uploaded asset are intended, as per discussion in Move BES upload check earlier, to prevent calling FindMissingBlobs #16999)isDirectory
andisSymlink
on an omitted file an error (they would otherwise returnfalse
, which is technically incorrect).This should improve performance. Benchmarking (in Bazel v6) has shown 54% of a Bazel profile for a fully cached build is spent in
readPathMetadata
.The test case in this PR appears to be the only such test for minimal mode (at least in Java code).
Fixes #20576
TODO
PathConverterImpl
the full list of files, while only a subset is uploaded under minimal they paths are still needed for protos.no-remote-cache
tagged)